Generally, a debugger is a tool, often a computer program, which is designed to help find errors in another program by allowing a user to step through the target program and examine data and check conditions relating to the execution of the program. This is done in order to allow analysis of a program in its running state, thereby providing a tool for troubleshooting problems and/or otherwise improving the performance of the program.
A debugger works by “attaching” to a particular process (the “debuggee” or “target” process) and, typically, by performing some or all of the following operations, either under the control of a user or in response to calls or conditions in the debuggee process:    (1) read and/or write process memory for the debuggee process;    (2) get and/or set the thread context for the debuggee process;    (3) control running threads of the debuggee process; and    (4) handle debug events that occur in the debuggee process.
The debugger can not directly perform these functions for or on the debuggee, since in general operating systems protect processes from each other, including their specific data, registers, control information, and any other like state of a given process. For example, operating systems do not generally allow one process to access process memory or thread context of another process. Instead of such direct access between threads, the kernel of a given operating system, or some other privileged software acting on behalf or instead of the kernel, mediates between the debugger process and debuggee process. For example, when a debugger process would like to get the context for a thread of the debuggee process, the kernel first receives the request, verifies that the debugger has the correct permissions to perform the requested action, and, if it does, provides the debugger with the requested portions of the debuggee process' thread context (for a read) or writes the requested data to the debuggee process' thread context (for a write).
Remote debugging is used when a debugger is run on a first system and the program being debugged (“debuggee” program) is running on a second system. In order to facilitate this remote debugging, a “debug stub” also runs on the second system. Remote debugging allows the debugger user interface to run on the first system in order to debug the debuggee program, which provides flexibility for the user of the debugger program where, for example, running the debugger program on the second system is not convenient or feasible. The debug stub serves as a surrogate for the debugger on the second system. When the debugger wishes to get the context for a thread of the debuggee process on the second system, for example, the request is passed to the debug stub, which communicates with the kernel on the second system and obtains the information.
However, it may be desirable to allow two execution environments (either in a single computing environment or in more than one computing environment) to interact with each other while preventing events at one of the execution environments from undermining the ability of another to behave as it is expected to behave. For example, two or more operating systems that run on a single machine are examples of execution environments that may need to interact with each other in a single computing environment. If this is the case, and one of the execution environments includes a high assurance component whose behavioral specification requires it not to “leak” information outside of itself, it can be appreciated that such a high assurance component (such as a high assurance operating system) cannot trust another kernel with the task of dispatching a thread, since the act of dispatching the thread would necessarily give the dispatching operating system access to the thread's register contents. Additionally, there are various other types of attacks to an operating system that are based on scheduling threads to run at inappropriate times (e.g., running threads that have been designated as not runnable, or running a thread that is already running).
In such a case, it may be desirable for a first operating system to make scheduling decisions about certain of the second (high assurance) operating system's processes (e.g., how to account for a thread's priority), but the ultimate dispatch of the second operating system's processes is performed by the second operating system—which can protect itself from an attack by refusing to perform the dispatch if conditions are not appropriate to run the thread. For processes that are to work in this bifurcated manner (“bifurcated processes”), a first “trusted agent” or “work process” is created in the second operating system, and a corresponding “shadow process” is created in the first operating system. Where processes include multiple threads, multiple bifurcated threads are created. The thread scheduler of the first operating system is used to schedule the threads of the trusted agent by scheduling the shadow process to run.
Whenever a thread of a shadow process is dispatched, the ultimate effect is to run the corresponding thread of the corresponding trusted agent in the second operating system. The mechanism by which the trusted agent is invoked is via the shadow process. When the shadow process is dispatched, a service exposed to shadow processes allows a change of context to the second operating system with a request that the corresponding trusted agent process be run. The trusted agent threads in the second operating system may be referred to as “work threads” because these are the threads that will actually perform the substantive work that needs to be done in the second operating system. These trusted agent threads are in contrast to the shadow threads, whose function is to be schedulable by the first operating system's scheduler so that they can invoke the trusted agent threads. In most cases, the second operating system can validate the appropriateness of such invocations; in this way, the use of the first operating system cannot cause the high assurance second operating system to behave in an unintended manner by incorrectly scheduling operating system's threads.
Shadow threads may be created by an agent of the second operating system that execute inside of the first operating system. These threads communicate with the second operating system. When the second operating system creates a thread, it informs the shadow process that the new thread has been created, and also informs the shadow process of the new thread's internal thread ID. The shadow process then instantiates a new shadow thread in the first operating system. The new shadow thread knows the thread ID that has been communicated to the shadow process. When the shadow thread runs and invokes a context switch back to the second operating system, the shadow thread uses the thread ID to instruct the second operating system which thread should be started.
In such a two execution environment context, because of the high assurance nature of one environment, it may not be advisable to allow a debugging process running on the first operating system to debug a target process which runs on the high assurance second execution environment. (Such a process running on a high assurance execution environment is termed a “trusted agent”). A debugger may not be available on the high assurance environment, or it may be inadvisable to allow the use of such a debugger to maintain the assured nature of the environment. Similarly, a debug stub may be unavailable for an environment or it may not be advisable to use a debug stub in an environment in order to maintain the assured nature of the environment. Where shadow threads are being used, the traditional methods of remote debugging with a debug stub can not be used, as each process is scheduled in one environment but the context and other process information is stored in the high assurance second execution environment. Thus the traditional methods of remote debugging can not be used.
In order to allow debugging to occur in the face of the high assurance environment running the target process, a debugger may be allowed to attach to the shadow process, and to receive scheduling information via the shadow process. Requests for any information relating to debugging which can not be gleaned from an examination of the shadow threads of the shadow process is passed to the high assurance environment. This concept is described in U.S. patent application Ser. No. 10/428,678, “User Debugger For Use On Processes Running In A High Assurance Kernel In An Operating System.”
However, in such a situation, when starting up debugging of a process there is no opportunity to stop the trusted agent process for debugging at the initiation of the trusted agent process. In addition, in order to collect information from the second, high assurance environment, the debugger must be able to provide information regarding the current state of the trusted agent.
A debugger, to function fully, must have the capability to cause execution of the threads of the trusted agent to stop. But, in the situation detailed above where a debugger attaches to the shadow process, when the threads of the trusted agent are stopped for debugging by stopping the shadow threads, for example when the trusted agent is halted or being stepped through, these threads can not provide the requested information regarding the thread context.
In view of the foregoing there is a need for a system that overcomes the drawbacks of the prior art.